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FOREWORD 


The Software Engineering Laboratory (SEL) is an organization 
sponsored by the National Aeronautics and Space Administration/ 
Goddard Space Flight Center (NASA/GSFC) and created for the 
purpose of investigating the effectiveness of software engi- 
neering technologies when applied to the development of appli- 
cations software. The SEL was created in 1977 and has three 
primary organizational members: 

NASA/GSFC (Systems Development and Analysis Branch) 

The University of Maryland (Computer Sciences Department) 
Computer Sciences Corporation (Flight Systems Operation) 

The goals of the SEL are (1) to understand the software devel- 
opment process in the GSFC environment; (2) to measure the 
effect of various methodologies, tools, and models on this 
process; and (3) to identify and then to apply successful de- 
velopment practices. The activities, findings, and recommen- 
dations of the SEL are recorded in the Software Engineering 
Laboratory Series, a continuing series of reports that in- 
cludes this document. A version of this document was also 
issued as Computer Sciences Corporation document 
CSC/TM-87/6017. ^ 

The contributors to this document include 

Sandra Perry, Leon Jordan, William Decker, Gerald Page 
(Computer Sciences Corporation) 

Frank McGarry, Jon Valett (Goddard Space Flight Center) 

Single copies of this document can be obtained by writing to 

Frank E. McGarry 
Code 552 
NASA/GSFC 

Greenbelt, Maryland 20771 
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ABSTRACT 


The product assurance policies and procedures necessary to 
support flight dynamics software development projects for 
Goddard Space Flight Center are presented. The quality assur 
ance and configuration management methods and tools for each 
phase of the software development life cycle are described, 
from requirements analysis through acceptance testing; mainte 
nance and operation are not addressed. 
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SECTION 1 - INTRODUCTION 


This document describes the product assurance policies and 
procedures necessary to provide quality products throughout 
the software development life cycle. It is intended for use 
in flight dynamics software development projects at Goddard 
Space Flight Center. For this document, product assurance 
consists of quality assurance and configuration management, 
both of which are essential for producing maintainable, com- 
plete, and reliable software product deliverables. 

Quality assurance is the process of reviewing all software 
development products to ensure that they meet or exceed a pre- 
defined set of guidelines and standards and are complete in 
content. It involves selecting guidelines and standards for 
the requirements, design, test, system description, user 
documentation, and code and checking that they are met.. 

Configuration management is the process of providing an 
orderly, systematic, and controlled flow of software develop- 
ment products in a changing environment. It involves estab- 
lishing product baselines and tracking changes to the 
baselines . 

This document is intended for use by flight dynamics software 
development technical managers, task leaders, and quality 
assurance and configuration management personnel. A technical 
manager is defined as the section manager or first-line super- 
visor responsible for the project. 

The purpose of this document is to define the policies and 
procedures for product assurance and the actions that will be 
taken. It also provides guidelines for developing quality 
assurance and configuration management plans. Each technical 
manager will tailor the plans to the specific project at hand. 
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This document is a high-level presentation, not an independent 
self-ref erence , and should be used in conjunction with the 
following publications: 

• Recommended Approach to Software Development (Refer- 
ence 1) , for the software development life-cycle 
phases and the activities and products resulting from 
each phase 

• Manager's Handbook for Software Development (Refer- 
ence 2), for planning guidance, standard document 
formats, and project monitoring guidance 

• Programmer's Handbook for Flight Dynamics Software 
Development (Reference 3), for programming standards 
and conventions 

• Software Verification and Testing (Reference 4), for 
testing procedures and code-reading techniques 

The document is structured as follows. Section 2 is an over- 
view of product assurance and its role in the life-cycle man- 
agement of software development projects. It presents the 
major goals of quality assurance and configuration management, 
describes general planning factors, and discusses product 
assurance products and processes. 

Sections 3 and 4 describe the policies and procedures for 
quality assurance and configuration management, respectively. 
They include the objectives and approach, methods and tools to 
be used, activities required during each phase of the software 
development life cycle, and management considerations. 

Appendixes A and B present guidelines for preparing quality 
assurance and configuration management plans, respectively. 
Appendix C contains quality assurance checklists, and Appen- 
dix D contains standard configuration management forms. 
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SECTION 2 - PRODUCT ASSURANCE OVERVIEW 


2.1 PRODUCT ASSURANCE GOALS 

The goal of product assurance is to produce maintainable, com- 
plete, and reliable software product deliverables through a 
combination of quality assurance and configuration management 
activities. Quality assurance addresses compliance with 
standards during the production of code and supporting docu- 
mentation. Configuration management addresses the management 
of change throughout the software development process. 

2.1.1 QUALITY ASSURANCE GOALS 

The goal of quality assurance is to ensure the maintainability 
and completeness of the software product deliverables result- 
ing from the development of flight dynamics software. This is 
achieved by identifying standards that are to be met or ex- 
ceeded in the development of the products. These standards 
are documented in the Software Engineering Laboratory series 
of documents and cover both the specific content and format 
for the requirements, design, test, system description and 
user documentation and the way in which the code will be de- 
veloped (e.g., top-down, structured, single function per mod- 
ule, uniform comment format). Another essential goal of 
quality assurance is to ensure that each software product ful- 
fills its specific intent in content. This is accomplished in 
the normal review process of all project deliverables. 

Quality assurance should be made an integral part of software 
development activities by initiating it early in the software 
development process. The standards, guidelines, and conven- 
tions that establish the ground rules for the development 
process are identified in the Quality Assurance Plan, normally 
developed as part of the Software Development /Management 
Plan. This plan is published and distributed to all develop- 
ment team members, and frequent spot checks and reviews are 
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made to eliminate doubts regarding the standards and formats 
to be used throughout the development process. 

For the purpose of this document, quality assurance is pri- 
marily directed toward the format and content of the software 
products and the standards used to develop them. Ensuring 
that the products represent and meet operational requirements 
is addressed through the normal review of preliminary and 
final documents and software by the development team members, 
task leader, and technical manager. Other methods used to 
ensure that operational requirements are met through the for- 
mal design review and testing processes (covered in Refer- 
ence 4) are not discussed in this document, except for the 
supporting product assurance activities for the formal reviews. 

2.1.2 CONFIGURATION MANAGEMENT GOALS 

The goal of configuration management is to produce reliable 
software product deliverables. This is achieved by providing 
for an orderly, systematic, and controlled development of the 
products in a changing environment. It involves identifying 
products that will be monitored for change management control, 
establishing the baseline for these products, and maintaining 
records that trace required changes. Changes are made within 
a formalized structure under controlled procedures. 

The configuration management goals and processes described in 
this document are concerned with product deliverables outside 
both the formal Configuration Control Board (CCB) process and 
hardware configuration management. The interfacing activities 
with the CCB are, however, included. 

2.2 PLANNING 

Quality Assurance and Configuration Management Plans are de- 
veloped at the beginning of each project and are usually 
included in the Software Development/Management Plan. The 
plans specifically identify the products to be managed by the 
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quality assurance and configuration management processes and 
the tools and methods to be used to carry out the processes. 
Identifying the individuals who will act as the quality as- 
surer and the configuration manager is also a key factor to be 
considered in the planning process. 

The plans reference the specific standards to be used for a 
project. Table 2-1 lists the usual software products and the 
major reference documents containing the standards and formats 
used in their production. These references are oriented to- 
ward software development using the FORTRAN programming lan- 
guage on either IBM or VAX computers. The standards presented 
in these references will be used unless deviations and substi- 
tutions are necessary. 

2.2.1 PLANNING FOR LEVEL OF DETAIL 

The level of detail expected to be covered in the product 
assurance process depends on the type of project: operational 

(e.g., attitude ground support systems), research and develop- 
ment (e.g., tools and analysis systems), or support (e.g., 
utilities); the criticality of the products; and the amount of 
effort allocated for product assurance activities. The tech- 
nical manager's primary guideline is to tailor the level of 
detail to the specific product. 

Common sense must be applied when determining the extent of 
activities to be conducted for each deliverable. Tradeoffs 
have to be considered regarding the cost of the product assur- 
ance activities and the value added by the activity in rela- 
tion to the specific product. 

Product assurance activities should apply only to product de- 
liverables, that is, not to transitory documents. In the 
quality assurance process, spot checks of code as it is ini- 
tially developed provide a less intense review than code- 
reading techniques; thorough reviews of documents before their 
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Table 2-1. 



• Software Development/ 
Management Plan 

• Requirements Analysis Summary 
Report 

• Preliminary Design Report 

• Detailed Design Document 

• Test Plans 

• User's Guide 

• System Description 

• Software Development History 



• Prologs 

• PDL 

• Code 

• System Delivery Tape 
Other Items 


• PDR Materials 

• CDR Materials 
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publication ensure their completeness and adherence to stand- 
ards. Configuration management procedures should be concerned 
with requirements changes that have major design/performance/ 
resource implications and all changes to controlled software, 
that is, they should not involve tracking the correction of 
typographical errors. 

2.2.2 PLANNING FOR PROJECT SIZE 

The technical manager also tailors the product assurance ac- 
tivities in the product assurance plans to the size of the 
project. The level of effort or formality of the activities 
will, however, always depend on the criticality of the soft- 
ware products and not necessarily on the size. General plan- 
ning guidance for size is as follows: 

• Large Projects 

More than 200K source lines of code 
Staff of more than 15 people 

Product assurance activities normally follow 
formal procedures and are conducted by special- 
ists on a full-time basis. Approximately 5 per- 
cent of the total project effort is devoted to 
product assurance activities. 

• Medium Projects 

50K to 200K source lines of code 
Staff of 8 to 15 people 

A separate, dedicated product assurance staff is 
normally not necessary, and the responsibility 
is assigned to one of the technical staff mem- 
bers on a part-time basis. Approximately 2 to 
5 percent of the total project effort is devoted 
to product assurance activities. Attitude 
ground support system projects normally fall in 
this category. 
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• Small Projects 

Less than 50K source lines of code 
Fewer than 8 people 

Product assurance activities are normally accom- 
plished by the task leader on the project and 
conducted formally or informally depending on 
the criticality of the project. Support and 
simulator projects fall into this category. 

These levels of effort do not include the effort spent by de- 
velopment team members and the technical manager in performing 
quality assurance duties that are part of the normal develop- 
ment process. For example, the effort spent by the developers 
reviewing the code as they develop and unit test or that spent 
in the normal documentation editing process are not included. 
This document describes what the product assurance process 
does include and what must be considered when establishing the 
level of effort for the project. 

2.3 PRODUCTS AND PROCESSES 

Figure 2-1 depicts the normal product assurance processes that 
are applied to software products and supporting documentation 
throughout the software development life cycle. Some products 
are placed under product assurance for only one phase of de- 
velopment and are then superceded by other products. For ex- 
ample, the design of the system is documented beginning with 
the Requirements Analysis Summary Report and is replaced in 
successive phases of development by the Preliminary Design 
Report; the Detailed Design Document; and finally by the Sys- 
tem Description, which is delivered with the system. Each of 
these documents is normally placed under configuration manage- 
ment until it is replaced by the next document in the series. 
Other products, such as the Software Development/Management 
Plan and the controlled software library, are placed under the 
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Figure 2-1. Items Typically Part of Product Assurance Process 



quality assurance and configuration management processes, re- 
spectively, for the duration of the development activity. 

To further amplify Figure 2-1, the role of product assurance 
in the software development life-cycle phases is described 
briefly in the following subsections. Reference 1 discusses 
the life-cycle phases in more detail. 

2.3.1 REQUIREMENTS ANALYSIS PHASE 

• Development Activities — Functional specifications are 
translated from mission terms into a software-supportable form 
and documented in the Requirements Analysis Summary Report, 
which is used as a basis for preliminary design. The final 
version of the Functional Specifications and Requirements Doc- 
ument is also completed, and a system requirements review (SRR) 
is held to evaluate the completeness of the requirements. The 
Software Development/Management Plan is also produced. 

• Product Assurance Activities — Quality assurance and 
configuration management activities are planned and documented 
in the Software Development/Management Plan. Quality assur- 
ance checks for completeness, content, and format are also 
made on the Software Development/Management Plan and the Re- 
quirements Analysis Summary Report. 

2.3.2 PRELIMINARY DESIGN PHASE 

• Development Activities — A software system architecture 
is defined based on the requirements given in the Functional 
Specifications and Requirements Document and the Requirements 
Analysis Summary Report. This architecture is translated into 
major functional subsystems and documented in the Preliminary 
Design Report. The functional design is formally presented 
for review at the preliminary design review (PDR) . Prelimi- 
nary design is considered complete when responses to the PDR 
comments and criticisms have been incorporated as change pages 
into the final Preliminary Design Report. 
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• Product Assurance Activities — Quality assurance 
activities include reviews of the Preliminary Design Report 
and the preliminary design review materials for completeness, 
and the Software Development/Management Plan for currency. 
Module PDL and prolog development are also spot checked and 
reviewed for standards compliance. Configuration management 
activities involve monitoring the question-and-answer process 
between the developers and the requirements team, recording 
PDR changes, and tracking them into the final Preliminary De- 
sign Report. 

2.3.3 DETAILED DESIGN PHASE 

• Development Activities — The system architecture de- 
fined in preliminary design is extended to the subroutine 
level, producing the "code-to" specifications for the system. 
Design specifications are documented in the Detailed Design 
Document. The detailed design is formally presented at the 
critical design review (CDR) . The design is considered com- 
plete when responses to the CDR comments and criticisms have 
been incorporated into the Detailed Design Document. 

• Product Assurance Activities — Quality assurance ac-. 
tivities include spot checks and reviews of the program pro- 
logs and PDL, reviews of the Detailed Design Document and the 
CDR materials for standards compliance and completeness, and a 
review of the Software Development/Management Plan for cur- 
rency. Configuration management activities involve tracking 
requirements changes and questions and answers between the 
requirements team and developers, recording CDR changes, and 
tracking the changes into the final Detailed Design Document. 

2.3.4 IMPLEMENTATION PHASE (CODE AND UNIT TESTING) 

• Development Activities — New modules are coded from the 
design specifications, old code is revised to meet new require- 
ments, each module is unit tested and integrated into the sys- 
tem, and integration testing is performed. A System Test Plan 
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is also produced. Implementation is considered complete when 
all code is integrated into the controlled software library. 

An independent acceptance test team prepares the Acceptance 
Test Plan. 

• Product Assurance Activities — Quality assurance ac- 
tivities include spot checks and reviews of the code, code 
reading to check for standards compliance, a review of the 
System Test Plan for completeness, and a review of the Soft- 
ware Development/Management Plan for currency. Configuration 
management activities involve tracking questions and answers 
and requirements changes into the Detailed Design Document. 

The controlled software library baseline is established at the 
beginning of this phase, and changes and additions are tracked 
to it . 

2.3.5 SYSTEM TESTING PHASE (SYSTEM INTEGRATION AND TESTING) 

• Development Activities — The integrated system is vali 
dated, and end-to-end system capabilities are tested according 
to the System Test Plan. System testing is considered com- 
plete when all tests specified in the System Test Plan have 
been executed successfully. The preliminary User's Guide and 
System Description documents are also produced. 

• Product Assurance Activities — Quality assurance ac- 
tivities involve the review of the preliminary User's Guide 
and System Description for standards compliance and content . 
and review of the Software Development/Management Plan for 
currency. Configuration management activities include track- 
ing questions and answers and tracking changes to requirements 
the Detailed Design Document, and the controlled software li- 
brary. Baselines are established for the preliminary User's 
Guide and System Description. 
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2.3.6 ACCEPTANCE TESTING PHASE 


• Development Activities — An independent acceptance 
test team tests the system to ensure that the software meets 
all requirements. After the successful completion of accept- 
ance testing, the final versions of the software and the 
system documentation are delivered to the customer and, if 
applicable, an operational readiness review (ORR) is held to 
evaluate the readiness of the system to support operations. 

• Product Assurance Activities — The final quality assur- 
ance review ensures that all project deliverables are complete. 
The controlled software library remains under configuration 
management control until the end of this phase, when the sys- 
tem delivery file is delivered. 
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SECTION 3 - QUALITY ASSURANCE 


The objective of quality assurance is to ensure that software 
products and supporting documentation meet or exceed the 
standards established for flight dynamics software, resulting 
in maintainable and complete software product deliverables. 
Early and thorough planning is essential, as is the applica- 
tion of quality assurance procedures early in the development 
of each new product throughout the development life cycle. 

The quality assurance role is shared by everyone on the devel- 
opment team. The responsibilities of the quality assurer are 
to keep the quality assurance process highly visible to all 
members of the development team, to ensure that the quality 
assurance milestones are met, and to maintain a record of the 
quality assurance activities, The technical manager is re- 
sponsible for identifying the standards and guidelines to be 
followed for the project;. the technical manager and the task 
leader are normally responsible for reviewing the documenta- 
tion for completeness, standards compliance, and content. 

Team members generally share code-reading tasks. The quality 
assurer normally conducts frequent spot checks of the code for 
standards compliance; the quality assurer and the task leader 
review the code. Figure 3-1 depicts the quality assurance 
process . 

The purpose of this section is to present the policies and 
procedures for performing quality assurance activities to sup- 
port flight dynamics software development. It is organized as 
follows: First, the quality assurance stage is set by pre- 

senting management considerations for planning and the quality 
assurance plan. Second, quality assurance methods are dis- 
cussed along with the tools that are candidates to support the 
quality assurance process. Third, the products that are nor- 
mally quality assured and the processes, that is, the applica- 
tions of the methods and tools to these products, are 
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Figure 3-1. The Quality Assurance Process 
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described. Fourth, all of the previous information is sum- 
marized in one page of quality assurance activities for each 
life-cycle phase of development. 

3.1 MANAGEMENT CONSIDERATIONS 

3.1.1 PLANNING 

Before the Quality Assurance Plan is developed, the amount of 
effort to be allocated for quality assurance activities must 
be determined. Planning considerations for the amount of • 
effort required for product assurance activities are discussed 
in Section 2.2. Another major decision is the selection of 
the person (or persons) responsible for the quality assurance 
process. This individual should be knowledgeable in quality 
assurance methods, tools, policies, and procedures and have a 
strong propensity to follow up as a matter of practice. The 
quality assurer must be someone who works- well with the tech- 
nical manager and who can be relied on to work independently 
to ensure that the quality assurance tasks are accomplished. 

3.1.2 QUALITY ASSURANCE PLAN 

The Quality Assurance Plan specifies the procedures to be ap- 
plied during each phase of the software development life cycle 
and assigns responsibility for quality assurance. This plan 
is normally developed as part of the Software Development/ 
Management Plan. The technical manager is responsible for 
this plan and tailors it to the specific project. The plan 
addresses the following: 

• Standards and conventions to be followed by the de- 
velopment team 

• Formats for document deliverables (References 1 and 
2) tailored to the specific project 

• Checklists (Appendix C) for evaluating standards com- 
pliance and product completeness 
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• Deviations from the policies and procedures as stated 
in this document and the rationale for these devia- 
tions 

• Specific products to be monitored by quality assur- 
ance procedures 

• Methodologies and tools to be used for each product 

• Quality assurance milestones based on the major mile- 
stones of the project and the product deliverables 
(Compliance with standards must be enforced in the 
early stages of software and document production. 

Spot checks and reviews for compliance must therefore 
be scheduled appropriately.) 

• Person (or persons) responsible for quality assurance 
of the project and those who will review and audit 
the products 

Appendix A presents guidelines for preparing a Quality Assur- 
ance Plan. 

3.2 QUALITY ASSURANCE METHODS AND TOOLS 
3.2J1 QUALITY ASSURANCE METHODS 

The primary methods used in the quality assurance process are 
as follows: 

• Spot checks 

• Reviews 

• Audits 

• Code reading 

Spot checks are short-notice reviews of software and documen- 
tation samples to monitor standards cpmpliance. Spot checks 
are normally made at the beginning of product development, 
that is, when a document is being drafted or when coding 
begins. The quality assurer performs short-notice spot checks 
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with each developer in the first few weeks of each new devel- 
opment activity and provides the developers with specific 
feedback on standards compliance. Followup spot checks are 
made with developers who have not complied with the stand- 
ards. Spot checks are made frequently, but normally on a 
limited amount of material and on an informal basis. 

Reviews are a formal process and are normally scheduled at 
major milestones throughout the development cycle. The prod- 
ucts specified for review are reviewed and certified for 
standards compliance, completeness, and content. Reviews are 
normally made by the technical manager, the task leader, and 
the quality assurer. The quality assurer records the comple- 
tion of the reviews to establish traceability. 

Audits are a formal review used for ensuring that proper prac- 
tices and procedures are being followed, for projects in 
trouble, and for critical products where there is no room for 
failure. Audits are normally performed by personnel independ- 
ent of the development activity or possibly from another orga- 
nization. Audits of the software products have not been 
specified in the procedural material that follows. The tech- 
nical manager specifies the use of audits in the Software 
Development/Management Plan when they are appropriate. Ref- - 
erence 2 (Section 7) describes the audit procedure. 

Code reading is a systematic procedure for reading the program 
to determine if it is correct with respect to its intended 
function and logic. During this process, the code reader also 
checks to see if the code complies with established stand- 
ards. The code-reading method is used after the code has been 
developed and compiled and before unit testing. Development 
team members are responsible for code reading, and the quality 
assurer is responsible for recording that code reading has 
been performed for each program. 
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Specific forms for recording actions and certification forms 
for signoff of a finished product are used and maintained by 
the quality assurer. Traceability records of quality assur- 
ance activities are also retained in a notebook. 

3.2.2 QUALITY ASSURANCE TOOLS 

The tools available for use in the quality assurance process 
are as follows: 

• Checklists 

• RXVP80 

• LOADMAP 

• FORTRAN Static Source Code Analyzer Program (SAP) 

The following tools are primarily used for configuration man- 
agement but are included in this section because they can be 
used to determine the completeness of the software components: 

• Module Management System (MMS) 

• Master File List (MFL) 

3 . 2 • 2 . 1 Checklists 

Checklists contain the list of items that must either be ac- 
complished or considered for successful completion of an ac- 
tivity. Appendix C contains the recommended quality assurance 
checklists, those normally used for standard development ac- 
tivities. Technical managers will develop appropriate check- 
lists for nonstandard development efforts that do not conform 
to the normal life-cycle phases or that require different 
products than are specified in this document. 

3. 2. 2. 2 RXVP80 

RXVP80 is a tool available on both the VAX and IBM computers 
for structural analysis of FORTRAN source code. It is applied 
to each program module and is used in conjunction with code 
reading to detect logical inconsistencies in the code. For 
example, it flags inconsistencies in data modes and types in 
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expressions, assignments, and calling sequences. It also 
checks for a consistent number of arguments in calling se- 
quences and identifies the use of variables before they are 
set. Reference 5 describes RXVP80 in more detail. 

RXVP80 is used during the implementation phase to determine 
inconsistencies in individual modules after they have been 
unit tested and before they are placed in the controlled soft- 
ware library. It is also used to determine integration incon- 
sistencies after the modules have been added to the library. 
RXVP80 runs are normally made for groups of modules after unit 
testing and on a periodic basis from the controlled software 
library. The frequency of runs is determined by the amount of 
library activity. Weekly or biweekly runs are the norm. 

The developers normally make RXVP80 runs on their own code 
from their individual libraries, and the configuration manager 
normally makes the runs from the controlled library. The 
quality assurer keeps track of the RXVP80 runs for each module 
in the controlled software library. 

3. 2. 2. 3 LOADMAP 

LOADMAP is a quality assurance utility available on the IBM 
computer that provides the following information for execution 
modules: a linkage-editor map, an alphabetic program unit 

list, an unreferenced program unit list, and cross-references 
of calls and references for all program units. LOADMAP can be 
used for completeness and consistency checks for execution 
modules. It is normally used every time a new or changed exe- 
cution module is created and is used by the individual creat- 
ing the execution module to quality assure it. 

3 . 2 . 2 . 4 * FORTRAN Static Source Code Analyzer Program 

SAP is a tool available on both the VAX and IBM computers for 
use during the implementation phase to analyze the code for 
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complexity. It should be used on groups of modules, for ex- 
ample, on all modules of a subsystem to identify module out- 
liers in terms of complexity. For example, the number of 
compound statements or the number of nesting levels within the 
group of modules can be compared. The developer can use this 
information to determine if there are modules that exceed the 
norm and can then analyze these modules to see if a more di- 
rect coding approach can be taken. Reference 6 describes SAP 
in more detail. 

The quality assurer keeps track of SAP usage for all program 
modules. The configuration manager normally runs- SAP because 
the runs are made using the controlled software library(s). 

3. 2. 2. 5 Module Management System 

MMS is a tool available on the VAX computer to build system 
execution modules. It provides descriptions of all components 
of the execution module and controls compiling, assembling, 
and creating object modules and generating an execution mod- 
ule. It obtains information specifying the lineage of each 
system component by examining the components' creation dates. 
System consistency can be assured based on the dates. For 
example, the execution module will automatically be recreated 
using new object code created at a later date than the previ- 
ous execution module. MMS is used primarily as a tool for 
configuration management, but it is also used to quality as- 
sure the execution modules and the system delivery file for 
completeness. Reference 7 describes MMS in more detail. 

MMS is used during the implementation phase when execution 
modules are initially created for integration testing. It is 
also used throughout the succeeding testing phases and for the 
system delivery file. The configuration manager will normally 
make the MMS runs because the basis for MMS is the source code 
residing in the controlled software library(s) . 
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3. 2. 2. 6 Master File List and Tape Generator 

The MFL, which is available on the VAX computer, is also pri- 
marily used for configuration management but can serve as a 
quality assurance completeness check for all files in the con- 
trolled software library. The MFL can be compared to previous 
lists to determine differences and completeness. Reference 3 
(Appendix J) discusses the MFL in more detail. 

The MFL is created when the controlled software library is 
first created and is updated through subsequent phases of de- 
velopment until the system delivery file is delivered. It is 
normally reviewed by the configuration manager on a periodic 
basis, every 1 or 2 weeks, for example, and the completeness 
check is made by the quality assurer or the configuration man- 
ager, whoever is specified. 

The tape generator is used with the MFL to create a system 
delivery tape automatically from the MFL at the end of the 
development cycle. Reference 3 (Appendix J) discusses the 
tape generator capability in more detail. Using both the MFL 
and the tape generator ensures a complete system delivery tape. 

3.3 QUALITY ASSURANCE PRODUCTS AND PROCESSES 

The primary products monitored in the quality assurance proc- 
ess are documents, supporting materials (e.g., PDR and CDR 
materials), and software. 

3.3.1 DOCUMENTS 

The documents monitored in the quality assurance process are 
as follows: 

• Software Development/Management Plan 

• Requirements Analysis Summary Report 

• Preliminary Design Report 

• Detailed Design Document 

• User's Guide 

• System Description 
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• Build Test Plan and Test Results 

• System Test Plan and Test Results 

• Software Development History 

The Acceptance Test Plan is not discussed in this document be- 
cause it is normally prepared by the requirements team; it is 
not the responsibility of the software developers.. Most of 
the documents noted above will be placed under configuration 
management for change management control. 

The User’s Guide and the System Description are produced ac- 
cording to the same schedule and, in this document, are dis- 
cussed together. (In some systems, they may be combined into 
one document . ) 

Quality assurance reviews of the Software Development/ 
Management Plan are scheduled periodically to ensure that this 
document is current. It is updated throughout the life cycle, 
particularly at major milestones or when events trigger a 
change in the management approach. Formal configuration man- 
agement procedures are not recommended for this document be- 
cause it is the manager's tool, not a project deliverable. 

The initial issue of this document is reviewed for complete- 
ness according to the standard formats presented in Refer- 
ences 1 and 2. The technical manager is responsible for this 
plan . 

Quality assurance reviews of the Build and System Test Results 
are made for completeness to ensure that all test results and 
any discrepancies are documented. 

The following procedures are used for quality assuring all of 
the other documents: 

1. The quality assurer distributes the standard format 
for the document (the standard formats are given in Refer- 
ences 1 and 2) to team members along with the design, coding, 
and test plan development assignments. 
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2. The quality assurer makes frequent spot checks early 
in the development of each document to determine if the stand- 
ard is being followed and to clarify any questions regarding 
format. Problems that cannot be resolved between the quality 
assurer and the developer are brought to the attention of the 
task leader or the technical manager for resolution, depending 
on the nature of the problem. 

3. The document is reviewed using the standard format 
and checklist before publication or distribution as follows: 
The task leader reviews the document for content, format, con- 
sistency, and grammar and works with the individual drafters 
to clarify or redo their specific areas of responsibility. 

This task may be shared with the technical manager, or the 
technical manager may review the document after the task 
leader has made his/her review. Formal document deliverables 
are normally passed to a technical publications staff for 
editing and formatting for publication. The technical manager 
is responsible for determining that the substance of the doc- 
ument has not been altered in this formal editing process. 

4. The technical manager and the quality assurer sign 
the document, thus certifying that it adheres to the standard 
and that the content is as expected. 

5. The quality assurer records in the quality assurance 
notebook that the document has been certified as meeting the 
standard . 

3.3.2 SUPPORTING MATERIALS 

The supporting materi-als monitored in the quality assurance 
process are as follows: 

• PDR materials 

• CDR materials 

The technical manager, the task leader, and the quality 
assurer are responsible for reviewing the materials supporting 
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the PDR and CDR. The materials normally consist of briefing 
materials and supporting documentation. The recommended 
standards for these materials are presented in Reference 1 
(Appendix A) . Reviews are made of the content and format for 
consistency of presentation and for typographical errors. The 
quality assurer records that the reviews have been made in the 
quality assurance notebook. 

3.3.3 SOFTWARE 

The software products monitored in the quality assurance proc- 
ess are as follows: 

• Software Prologs, PDL, and code 

• System delivery file 

The procedures for quality assuring the code are as follows: 

1. The quality assurer makes frequent spot checks of the 
module prologs and PDL during the design phases and of the 
code (e.g., source code, job control language (JCL), panels, 
Graphic Executive Support System (GESS) displays) during the 
implementation phase. These checks are to ensure that the 
developers are practicing good habits using specified stand- 
ards. These standards are presented in Reference 3 for 
FORTRAN development projects. When problems occur that cannot 
be resolved between the quality assurer and the developer, 
they are brought to the task leader or the technical manager 
for resolution, depending on the nature of the problem. The 
quality assurer records when the spot checks are made in the 
quality assurance notebook. 

2. Development team members perform code reading on each 
coded module after it is compiled. This is used as a further 
check for standards compliance as well as for logic and con- 
tent. The quality assurer records that each module has been 
code read and by whom. 
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3. RXVP80 , if used, is run for groups of FORTRAN coded 
modules, normally by the individual developers, before the 
code is added to the controlled software library. 

4. RXVP80, if used, is also run on FORTRAN modules as 
they are integrated into the controlled software library or 
libraries. The configuration manager usually makes these runs 
and passes information regarding them to the quality assurer 
for recording in the quality assurance notebook. Other qual- 
ity assurance tools, such as SAP and LOADMAP, may also be 
run. Running these tools is normally the responsibility of 
the configuration manager because they are run from the con- 
trolled software library. If their use is to be tracked for- 
mally by the quality assurer, he/she must be informed that 
they are being used and must follow up to ensure that the re- 
sults are used to improve the quality of the software. 

5. Periodic reviews are also conducted at project mile- 
stones (e.g., at the completion of each build/release of the 
software) , using the standard as found in Reference 3 and the 
checklists as found in Appendix C, to ensure that all stand- 
ards have been met. Reviews are normally made by the task 
leader and are recorded by the quality assurer. 

The procedures for quality assuring the system delivery file 
are as follows: The file is checked for completeness in ac- 

cordance with the standards presented in the Quality Assurance 
Plan. Building the system delivery file at the end of the 
project is normally a team effort of the task leader, quality 
assurer, and configuration manager. The configuration manager 
is involved because the system source code resides in the con- 
trolled software library. Tools such as MMS, the MFL, and the 
tape generator can be used, if available, to aid in the com- 
pleteness test. The task leader is responsible for certifying 
that the system delivery file is complete, and the quality 
assurer records this action. 
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3.4 QUALITY ASSURANCE LIFE-CYCLE-PHASE SUMMARY 


The major quality assurance actions for each development phase 
are summarized in the following subsections. The standards, 
checklists, methods, and tools to be used for quality assuring 
each development product (both code and supporting documenta- 
tion) will be specified in the Quality Assurance Plan of the 
Software Development/Management Plan. The recommended stand- 
ards for each product are referenced in Table 2-1. 
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3.4.1 QUALITY ASSURANCE IN THE REQUIREMENTS ANALYSIS PHASE 

(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• Software Development/Management Plan 

• Requirements Analysis Summary Report 

Actions 

• Develop Quality Assurance Plan in Software 
Development/Management Plan (TM) 

• Software Development/Management Plan 

Quality assure against checklist and standard 
and certify complete (TM, TL) 

Record publication date and certification (QA) 

• Requirements Analysis Summary Report 

Review for completeness, content, and standards 
(TL, TM) and certify (QA) 

Record publication date and certification in 
quality assurance notebook (QA) 
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3.4.2 QUALITY ASSURANCE IN THE PRELIMINARY DESIGN PHASE 

(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• Preliminary Design Report 

• PDR materials 

• Software Development/Management Plan 

• Module prologs and PDL 

Actions 

• Distribute Preliminary Design Report standard formats 

to all team members (QA) when design assignments are 
made 

• Preliminary Design Report 

Spot check draft material using checklist and 
standard, sampling each developer's material (QA) 

Review drafts for completeness and content using 
checklist and standard (TM, TL) 

Certify and record publication date of document 
in quality assurance notebook (QA) 

• Review PDR materials for completeness using standard 
(TM, TL, QA) 

• Software Development/Management Plan 

Review for currency with PDR results (TM) 

Review updates, if any, for content (TM) 

Record update and quality assurance certifica- 
tion (QA) 

• Module prologs and PDL 

Spot check each developer's efforts for stand- 
ards compliance (QA) 

Review all at end of phase to ensure they are 
complete and comply with standards (TL) and 
record review in quality assurance notebook (QA) 
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3.4.3 QUALITY ASSURANCE IN THE DETAILED DESIGN PHASE 

(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• Detailed Design Document 

• CDR materials 

• Software Development/Management Plan 

• Module prologs and PDL 

Actions 

• Distribute Detailed Design Document standard formats 

to all team members (QA) when design assignments are 
made 

• Detailed Design Document 

Spot check draft material using checklist and 
standard, sampling each developer’s material (QA) 

Review drafts for completeness and content using 
checklist and standard (TM, TL) 

Certify and record publication date of document 
in quality assurance notebook (QA) 

• Review CDR materials for completeness using standard 
(TM, TL, QA) 

• Software Development/Management Plan 

Review for currency with CDR results (TM) 

Review updates, if any, for content (TM) 

Record update and quality assurance certifica- 
tion (QA) 

• Module prologs and PDL 

Spot check each developer’s efforts for stand- 
ards compliance (QA) 

Review all at end of phase to ensure they are 
complete and comply with standards (TL) and 
record review in quality assurance notebook (QA) 
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3.4.4 QUALITY ASSURANCE IN THE IMPLEMENTATION PHASE 


(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• Source code 

• Test plans and results 

• Draft User's Guide and System Description 

• Software Development/Management Plan 

Actions 

• Publicize and distribute standards for code produc- 

tion, test plans, and User's Guide and System De- 
scription (QA-) 

• Source code 

Conduct frequent spot checks using checklist and 
standard, sampling all developers’ code; record 
in quality assurance notebook (QA) 

Keep track of code reading of each module and 
use of any quality assurance tools (QA) 

•Review all source code for standards compliance 
and completeness, normally at completion of each 
software build/release (TL) , and record (QA) 

• Test plans and results 

Review build/release and system test plans for 
completeness using checklist and standard (TM, 

TL) 

Record reviews in quality assurance notebook (QA) 

Review test results for completeness and docu- 
mented discrepancies (TL, TM) and record (QA) 

• Draft Users 's Guide and System Description 

Review draft material at end of each build/ 
release for standards compliance (TL, TM) 

Record reviews in quality assurance notebook (QA) 

• Software Development/Management Plan 

Review for currency and update if necessary (TM) 
Record update and certify (QA) 



3.4.5 QUALITY ASSURANCE IN THE SYSTEM TESTING PHASE 


(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• System test results 

• Preliminary User's Guide and System Description 

• Software Development/Management Plan 

Actions 

• Review system test results for completeness and en- 
sure that all discrepancies are documented (TL, TM) 
and record (QA) 

• Preliminary User's Guide and System Description 

Review for standards compliance, completeness, 
and content (TL, TM) 

Certify and record that document fulfills qual- 
ity assurance requirements (QA) 

• Software Development/Management Plan 

Review for currency and update if necessary (TM) 
Record update and certify (QA) 




3.4.6 QUALITY ASSURANCE IN THE ACCEPTANCE TESTING PHASE 


(QA=Quality Assurer, TL=Task Leader, TM=Technical 
Manager) 

Products 

• User's Guide and System Description 

• Software Development History 

• System delivery file 

Actions 

• User's Guide and System Description 

Review for standards compliance, completeness, 
and content (TL, TM) 

Certify and record that document fulfills qual- 
ity assurance requirements (QA) 

• Software Development History 

Review for standards compliance, completeness, 
and content (TL, TM) 

Certify and record that document fulfills qual- 
ity assurance requirements (QA) 

• System delivery file 

Review for completeness using checklist, stand- 
ard, and available quality assurance tools (TL, 
QA) 

Certify that system delivery file is complete 
and ready for delivery (QA) 



SECTION 4 CONFIGURATION MANAGEMENT 


The objective of configuration management is to provide a sys- 
tematic method of controlling, tracking, and incorporating 
change into software products and supporting documentation 
throughout the software development life cycle. Change is 
managed to ensure the integrity, consistency, and reliability 
of the software products. Changes occur in two different 
ways: (1) through the natural evolution of the software de- 
velopment activity, in which functional specifications are 
translated into a preliminary design, further refined into a 
detailed design, and finally developed into code; as the sys- 
tem grows; and as errors are detected and corrected during the 
testing phases and (2) as a result of external influences such 
as a change in the environment or requirements. This latter 
type of change is normally unplanned, for example, when the 
planned launch of a satellite from the Space Transportation 
System is. changed to an unmanned vehicle. The payload is re- 
duced, which results in a change to the satellite's flight 
dynamics characteristics. These changes would dictate major 
design changes to an attitude ground support system. 

Managing evolutionary change is necessary in ensuring that the 
functionality of the system is consistent with system require- 
ments and faithfully represents those requirements as the de- 
velopment proceeds. Evolutionary change is managed through 
the following processes: PPR and CDR; system walk-throughs; 

the interaction of the requirements team and the developers; 
release and system testing and software error correction; and 
acceptance testing, which is devoted to testing the function- 
ality of the system against operational requirements. The 
configuration management procedures described in this document 
support the control of evolutionary change by tracking 
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(1) questions and answers between the requirements team and 
the developers, (2) changes resulting from the PDR and CDR , 
and (3) changes and additions to the controlled software 
library. 

Managing unplanned, environmentally induced changes involves 
establishing appropriate product baselines and tracking and 
controlling changes to these baselines. Major requirements 
changes to the system design or to performance that result in 
changes to external interfaces or that increase system re- 
sources are tracked to the baselines. These are the types of 
changes that are not made unilaterally by the development 
team, but are reviewed and approved by a higher authority. 

For operational systems that support the Flight Dynamics Fa- 
cility, changes are normally tracked by the Code 55Q Configu- 
ration Management Office (CMO) in support of the Flight 
Dynamics Facility Configuration Control Board (CCB) . 

There is a liaison between the configuration manager's duties 
as described here and the formal CCB procedures and the CMO. 
The configuration manager tracks changes resulting from the 
PDR and CDR and from major requirements changes and provides 
the necessary interface with the CMO. 

Configuration management is a team effort. The configuration 
manager's responsibility is to keep track of all appropriate 
changes, not to identify the changes that need to be tracked. 
The latter responsibility falls to the technical manager, the 
task leader, the requirements team, and the PDR and CDR proc- 
ess. The procedures for conducting configuration management 
must therefore be highly visible to all development and re- 
quirements personnel at the beginning of the project and when 
new personnel join the project. Figure 4-1 depicts the con- 
figuration management process. 

The purpose of this section is to present the policies and 
procedures for performing configuration management activities 
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Figure 4-1. The Configuration Management Process 
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to support flight dynamics software development. It is or- 
ganized as follows: First, the configuration management 

stage is set by presenting management considerations for 
planning and the configuration management plan. Second, 
configuration management methods are discussed along with 
the tools that are candidates to support the configuration 
management process. Third, the products that are normally 
configuration managed and the processes, that is, the appli- 
cations of the methods and tools to these products, are de- 
scribed. Fourth, all of the previous information is 
summarized in one page of configuration management activi- 
ties for each life-cycle phase of development. 

4.1 MANAGEMENT CONSIDERATIONS 

4.1.1 PLANNING 

Planning for configuration management activities is one of the 
initial steps in the software development life cycle and is 
documented in the Configuration Management Plan as part of the 
Software Development/Management Plan. Early planning is nec- 
essary for configuration management procedures to be well es- 
tablished and integrated throughout all phases of the software 
development activities supporting the Flight Dynamics Facility. 

The major planning decisions to be made before the Configura- 
tion Management Plan is developed involve identifying the 
products and product baselines that will be managed and the 
.types of changes that will be tracked to the product base- 
lines. Another key decision is the selection of the person 
(or persons) responsible for the configuration management 
process . 

Planning considerations for the amount of effort required for 
configuration management activities are discussed in Sec- 
tion 2.2. The process described in this document is well 
established for attitude ground support system development 
efforts on IBM hardware. For this type of system, the actions 
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described for maintaining change records are necessary to sup- 
port the Code 550 CMO and the Flight Dynamics Facility CCB and 
to maintain a controlled software library. 

For other types of projects, such as experimental or research 
projects, the types of documents and document baselines de- 
scribed here may not be produced, and the CCB may not be in- 
volved. In those cases, the technical manager must identify 
the documents to be monitored and the level of effort to be 
expended managing changes to them. Decisions will be made 
based on the following considerations: Is the document a de- 

liverable? How critical is it to the overall development ef- 
fort? Are only changes of substance to be tracked, or are all 
changes to be tracked, including typographical error correc- 
tions and statements of. clarification? The general guideline 
is to track changes in requirements that affect performance, 
changes in design that affect system resources, and any 
changes to external interfaces. Controlled software libraries 
are normally established as baselines for all project types, 
and changes are always tracked to the libraries. 

The person selected as the configuration manager should be 
knowledgeable in the configuration management process and also 
have the tenaciousness to follow up as a matter of practice. 
Followup is essential in change management to ensure that 
every tracked change is incorporated into the appropriate 
product. The configuration manager is responsible for keeping 
the configuration management methods, tools, and milestones 
visible throughout the development cycle. Identifying a sys- 
tem librarian to maintain the controlled software library will 
be the minimum control point of any development effort, re- 
gardless of the size of the project. The librarian will main- 
tain control of the software throughout the system development 
activity. 
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4.1.2 CONFIGURATION MANAGEMENT PLAN 


The Configuration Management Plan specifies the procedures to 
be applied to the software and specific documents to ensure a 
controlled and systematic software development effort. The 
plan also assigns responsibility for configuration manage- 
ment. The .technical manager is responsible for this plan and 
tailors it to the specific project. The plan addresses the 
following : 

• Product baselines to be placed under configuration 
management control 

• Changes to be tracked to each baseline 

• Approval, authority for each change 

• Specific procedures to be followed by all team mem- 

bers for each product baseline 

• Methodologies/tools to be used 

• Identification of change forms to be used in relation 
to specific products 

• Location of all change forms for central reference 

• Identification of the configuration manager 

• Procedures for dealing with external interfaces 

Code 550 CMO/CCB 
Requirements team 

Appendix B presents guidelines for preparing a Configuration 
Management Plan. 
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4.2 CONFIGURATION MANAGEMENT METHODS AND TOOLS 


4.2.1 CONFIGURATION MANAGEMENT METHODS 

The primary methods used in the configuration management proc- 
ess are as follows: 

• Establishing product baselines and specifying the 
types of changes to be tracked to each baseline 

• Tracking changes and maintaining change records 
against the product baselines 

Product baselines are established at specific points in the 
software development activity. Baselines represent a state 
of completeness or a culmination of activities, a formal de- 
parture point for controlling future changes. For example, 
the Preliminary Design Report is established as a baseline at 
the time of the PDR, when it represents the culmination of the 
preliminary design phase. Changes are tracked to this report 
until the Detailed- Design Document baseline is established. 

The product baselines that are normally established for con- 
figuration management control are as follows: 

• Preliminary Design Report 

• Detailed Design Document 

• User’s Guide and System Description 

• Controlled Software Library — Prologs, PDL, and code 

The types of changes that are tracked for the product base- 
lines are major requirements changes normally requiring formal 
CCB review and approval for operational systems supporting the 
Flight Dynamics Facility and all changes to controlled soft- 
ware libraries. 

Change records are maintained to keep track of change activ- 
ity. They are the responsibility of the configuration manager 
and are maintained in a central location in a configuration 
management notebook so that they are accessible to all devel- 
opment personnel. 
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4.2.2 CONFIGURATION MANAGEMENT TOOLS 


The tools normally used in the configuration management proc- 
ess are as follows: 


• Configuration Management Forms 

• Configuration Analysis Tool (CAT) 

• Controlled Software Library Tools — PANVALET/PANEXEC/ 
PANCOMPARE, Code Management System (CMS), CONMAN 

• Module Management System (MMS) 

• Master File List (MFL) 

• LISTIDR 

4. 2. 2.1 Configuration Management Forms 


A series of forms is used to track changes to flight dynamics 
software development activities. Appendix D contains the 
change forms that are described in this section. These forms 
are commonly used for configuration management control of op- 
erational development efforts such as attitude ground support 
systems. They are not intended to be all inclusive, and the 
technical manager must develop additional forms or tailor 
existing forms for a specific project. 

Change forms are used to document questions resulting from 
PDRs and CDRs regarding requirements, design, and system per- 
formance. The Review Item Disposition (RID) is used to record 
the questions and the answers. The configuration manager is 
responsible for ensuring that the questions and answers are 
documented and forwarded to the CMO and that any resulting 
changes are made to the appropriate design document. 

The Requirements/Specifications Question and Answer Form is 
used for questions between the system developers and the re- 
quirements team. These individuals complete the form, and the 
configuration manager ensures that the questions are an- 
swered. Any question and answer that results in a major 
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design/performance change is formally tracked for CMO/CCB 
action. 

Requirements changes are documented on the Configuration 
Change Request (CCR), which is submitted to the Code 550 CMO 
and then passed to the CCB for disposition. The configuration 
manager records that a CCR has been filled out and tracks the 
changes that are approved to the appropriate product. 

If the change is approved, a Document Change Notice (DCN) is 
prepared, and the configuration manager tracks and records the 
final disposition of the change in the appropriate document. 
The question-and-answer process goes on throughout the devel- 
opment activity. 

A Component Origination Form (COF) is completed by the devel- 
opers for each module that has been coded, code read, and unit 
tested before it is added to the controlled software library. 
These forms are used during the implementation phase and are 
submitted to the project librarian/configuration manager be- 
fore modules are added to the controlled software libraries. 

A Change Report Form (CRF) is also completed by the developers 
whenever a change is made to any module in the controlled 
software library or libraries. Changes are not made unless 
the change request is approved, normally by the technical man- 
ager, and submitted to the configuration manager, who makes 
the changes to the library. 

4. 2. 2. 2 Configuration Analysis Tool (CAT) 

The Configuration Analysis Tool, available on both the VAX and 
IBM computers, is used to track and report project status. 

CAT is used to keep track of the following: discrepancies and 

changes, specification modifications, questions between the 
requirements team and the developers, and the RID forms. CAT 
provides management reports on each of these and is used 
throughout the development cycle. The configuration manager 
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normally provides the input for CAT, and the reports are used 
by the configuration manager as a followup tool and by the 
technical manager as a project status indicator. References 8 
and 9 discuss CAT in more detail. 

4. 2. 2. 3 Controlled Software Library Tools 

PANVALET, PANVALET/COMPARE, and PANEXEC are library management 
tools available on the IBM computer. PANVALET allows control- 
led access to source code libraries and maintains information 
about each member of the library (e.g., date of last access to 
a member, date of last maintenance of a member, and level num- 
bers that indicate how many times the member has been changed 
and identify specific statements that were changed within the 
member) . PANVALET/COMPARE provides a way of determining dif- 
ferences between two source code modules by performing soft- 
ware comparisons and generating reports on these comparisons. 
PANEXEC allows controlled access to object and execution mod- 
ule libraries and provides cross-referencing between source 
and execution modules. PANVALET is the primary tool of the 
configuration manager and is used to initiate the controlled 
library baseline at the beginning of the implementation 
phase. It is used until the system - delivery file is delivered 
at the end of the project. Reference 10 d-iscusses PANVALET in 
more detail. 

The Code Management System (CMS) is a source library control 
system for the VAX computer. Its use is similar to PANVALET. 
References 11 and 12 describe CMS in more detail. 

CONMAN is a library support tool available on the VAX and PDP 
computers that compares two libraries and produces a list of 
differences between them. It is used to compare the con- 
trolled version of files listed in a Master File List (see 
Section 3.2.2. 6) with an updated version. It is also used to 
produce command procedure files that will move only the 
changed and new software components into a controlled area. 
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CONMAN is used during the implementation phase when updates 
are made to the controlled software library and in subsequent 
phases until the system delivery file is created and delivered 
to the customer. 

4. 2. 2. 4 Module Management System 

MMS is a tool available on the VAX computer that can be used 
to track the lineage of system execution modules (see Sec- 
tion 3. 2. 2. 5). MMS should be used to keep track of changes to 
execution modules when they are initially built from the con- 
trolled software library(s) during the implementation phase. 
MMS reports are kept by the configuration manager in a central 
location for reference by all team members. 

4. 2. 2. 5 Master File List 

The MFL , as described in Section 3. 2. 2. 6, is used to facili- 
tate the tracking of differences between execution modules. 

4. 2. 2. 6 LISTIDR 

LISTIDR is a program available on the IBM computer that pro- 
vides an update history of execution modules. This program 
can be used to keep track of changes to the system's execution 
modules throughout the implementation phase and the subsequent 
test phases of development. LISTIDR is used when execution 
modules are first built from source code residing on the con- 
trolled software library or libraries. The individual creat- 
ing the execution module runs LISTIDR, but the results are 
passed to the configuration manager to be maintained in a cen- 
tral location for reference. 
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4.3 CONFIGURATION MANAGEMENT PRODUCTS AND PROCESSES 


The products that will normally be established for configura- 
tion management control are as follows: 

• Documents 

Preliminary Design Report 

Detailed Design Document 

User’s Guide and System Description 

• Software — Controlled software library, prologs, PDL, 

and codfe 

• Other Items 

Requirements/design decisions that remain to be 
determined 

Questions and answers between the requirements 
team and the developers 

4.3.1 DOCUMENTS 

4. 3. 1.1 Preliminary Design Report 

This document is established as a baseline at the time of the 
PDR during the preliminary design phase of the development 
effort. It remains under configuration control until the De- 
tailed Design Document baseline is established. The specific 
procedure for change management of this document is as follows 

The configuration manager maintains the list of changes to be 
made to the document in response to comments from the PDR. 

The configuration manager is responsible for ensuring that 
appropriate RIDs are filled out for all items identified dur- 
ing the PDR and that they are submitted to the Code 550 CMO 
for disposition by the CCB. Approved changes are then docu- 
mented with a DCN and produced as change pages to the docu- 
ment. The configuration manager is responsible for ensuring 
that change pages are distributed to the appropriate individ- 
uals. Through this process, the configuration manager keeps 
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track of each RID and each resulting change to ensure that all 
are accounted for. 

4. 3. 1.2 Detailed Design Document 

This document is established as a baseline at the time of the 
CDR during the detailed design phase of the development 
effort. It remains under configuration control until the Sys- 
tem Description Document is established as the baseline. 
Changes as a result of the CDR are tracked as described above 
for the Preliminary Design Report. 

4. 3. 1.3 User's Guide and System Description 

These documents, or a single document that includes both, are 
established as baselines when they are delivered as prelimi- 
nary documents in the system testing phase of development. 
Although these documents are normally drafted during the im- 
plementation phase, they are not placed under configuration 
control until they become deliverables as preliminary docu- 
ments in the system testing phase. They remain under config- 
uration control until they are delivered at the end of the • 
project. 

4. 3. 1.4 Tracking Requirements Changes to Documents Outside 
the PDR/CDR Process 

Requirements changes or major changes to system design or per- 
formance requirements outside the PDR/CDR process are docu- 
mented with a CCR . The form is filled out by the requirements 
team, the technical manager, or the task leader. It is sub- 
mitted to the CMO by the configuration manager and tracked by 
the configuration manager until the disposition of the change, 
normally as change pages into the Preliminary Design Report, 
the Detailed Design Document, or the System Description base- 
line, depending on the phase of development. 
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4.3.2 SOFTWARE 


Software libraries initiated during the preliminary design 
phase are used to establish the controlled baseline at the 
beginning of the implementation phase, normally using a li- 
brary management system such as PANVALET for the IBM computer 
or CMS for the VAX computer. When established, the controlled 
baseline contains reusable modules. 

Tracking changes to the controlled software library is the 
major activity of the configuration manager during the imple- 
mentation phase of development as each newly coded module is 
added to the library. Each component is documented on a COF 
before it is added to the library. As each module is integra- 
tion tested from execution load modules built from the con- 
trolled library, any errors detected during testing and their 
solution are documented on a CRF and approved by the task 
leader or the person designated in the Configuration Manage- 
ment Plan. When the change is approved, the configuration 
manager makes the changes to the library and maintains a copy 
of all CRFs . The configuration manager is the only person 
with write access to the controlled library, except for appro- 
priate personnel (e.g., the task leader) for backup when the 
configuration manager is not available. 

The configuration manager must also maintain sufficient backup 
files of the controlled software library, not only for insur- 
ance if the current library is destroyed, but also to be used 
when there is a problem in building execution modules from 
corrected code. If an error occurs that prohibits the crea- 
tion of the execution module, the previous version of the li- 
brary must be restored and then used to build execution 
modules for testing and subsequent updating. 

Specific procedures will have to be established based on the 
specific project for generating object code and execution mod-- 
ules from the controlled library, for determining where error 

4-14 


0389 



on the user's li- 


corrections will be made and tested (e.g., 
brary) before the controlled library is changed, for regres- 
sion testing when needed, and for the controlled software 
library’s file backup. 

CONMAN, if used, will account for and track changes between 
succeeding controlled software, libraries. 

4.3.3 OTHER ITEMS 

The configuration manager also tracks requirements or design 
decisions that remain to be determined and questions and an- 
swers between the requirements team and the developers. These 
items are part of the evolutionary development of a software 
development project. They generally clarify ambiguous re- 
quirements or design materials and sometimes result in major 
changes to the system. 

The configuration manager uses CAT, if it is specified for use 
in the Configuration Management Plan, to list and record the 
questions and answers and progress on the requirements or de- 
sign decisions that have not been determined. The configura- 
tion manager gathers the information for entry into CAT and 
distributes the CAT reports, and also follows up periodically 
on the decisions that remain and on any questions that are 
unanswered for more than 2 weeks. Questions and answers that 
result in major requirements changes, such as those affecting 
system performance, are processed as described in Sec- 
tion 4 . 3 . 1 . 4 . 

4.4 CONFIGURATION MANAGEMENT LIFE-CYCLE-PHASE SUMMARY 

The configuration management actions that are taken in each 
phase of the software development life-cycle are summarized in 
the following subsections. 
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4.4.1 CONFIGURATION MANAGEMENT IN THE REQUIREMENTS ANALYSIS 
PHASE 

(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager) 


Baselines Established 


Actions 


None 


Develop Configuration Management Plan in Software 
Development/Management Plan (TM) 

Establish configuration management procedures (change 
folders, change forms, etc.) and publicize to all 
team members (CM) 


a 
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4.4.2 CONFIGURATION MANAGEMENT IN THE PRELIMINARY DESIGN PHASE 


(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager) 

Baselines Established 

• Preliminary Design Report 
Actions 

• Track requirements changes 

Document with CCR form (TM, TL, or member of 
requirements team) 

Submit form to CMO (CM) 

Track change if approved into Preliminary Design 
Report (CM) 

• Track questions and answers between requirements team 
and developers 

Submit all questions and answers through CM 

Record question and date asked, and date an- 
swered when available, normally using CAT (CM) 

Follow up on any question unanswered for more 
than 2 weeks (CM) 

Track any resulting requirements changes from 
this process as described above 

• Preliminary Design Report 

Establish preliminary document used as basis for 
PDR as baseline (TM) 

Monitor RID form completion and disposition as a 
result of PDR (CM) 

Serve as interface with CMO (CM) 

Track changes resulting from approved RIDs to 
final report (CM) 



4.4.3 CONFIGURATION MANAGEMENT IN THE DETAILED DESIGN PHASE 

(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager) 

Baselines Established 

• Detailed Design Document 
Actions 

• Track requirements changes 

Document with CCR form (TM, TL, or member of 
requirements team) 

Submit form to CMO (CM) 

Track change if approved into Detailed Design 
Document (CM) 

• Track questions and answers between requirements team 
and developers 

Submit all questions and answers through CM 

Record question and date asked, and date an- 
swered when available, normally using CAT (CM) 

Follow up on any question unanswered for more 
than 2 weeks (CM) 

Track any resulting requirements changes from 
this process as described above’ 

• Detailed Design Document 

Establish preliminary document used as basis for 
CDR as baseline (TM) 

Monitor RID form completion and disposition as a 
result of CDR (CM) 

Serve as interface with CMO (CM) 

Track changes resulting from approved RIDs to 
final report (CM) 
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4.4.4 CONFIGURATION MANAGEMENT IN THE IMPLEMENTATION PHASE 

(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager, DT=Development Team Member) 

Baselines Established 

• Controlled software library 
Actions 

• Track requirements changes 

Document with CCR form (TM, TL, or member of 
requirements team) 

Submit form to CMO (CM) 

Track change if approved into Detailed Design 
Document (CM) 

• Track questions and answers between requirements team 

and developers 

Submit all questions and answers through CM 

Record question and date asked, and date an- 
swered when available, normally - using CAT (CM) 

Follow up on any question unanswered for more 
than 2 weeks (CM) 

Track any resulting requirements changes from 
this process as described above 

• Establish controlled software library baseline, com- 
posed of reusable code identified in previous two 
design phases (CM) 

• Track changes to controlled software library 

Prepare COF or equivalent and submit to CM for 
each newly coded module after it has been unit 
tested for addition to library (DT) 

Prepare CRFs for changes to coded modules to 
correct errors detected in integration testing 
and submit to CM (DT) 

Update library with change indicated on CRF and 
retain copies of CRFs (CM) 

Provide appropriate number of backup copies of 
controlled software library (CM) 
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4.4.5 CONFIGURATION MANAGEMENT IN THE SYSTEM TEST PHASE 

(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager, DT=Development Team Member) 

Baselines Established 

• User's Guide and System Description 
Actions 

• Track requirements changes 

Document with a CCR form (TM, TL, or member of 
requirements team) 

Submit form to CMO (CM) 

Track change if approved into Detailed Design 
Document or User's Guide and System Description 
when baseline is established (CM) 

• Track questions and answers between requirements team 
and developers 

Submit all questions and answers through CM 

Record question and date asked, and date an- 
swered when available, normally using CAT (CM) 

Follow up on any question unanswered for more 
than 2 weeks (CM) 

Track any resulting requirements changes from 
this process as described above 

• Track changes to the controlled software library 

Prepare COF or equivalent and submit to CM for 
each newly coded module after it has been unit 
tested for addition to library (DT) 

Prepare CRFs for changes to coded modules to 
correct errors detected in system testing and 
submit to CM (DT) 

Update library with change indicated on CRF and 
retain copies of CRF (CM) 

Provide appropriate number of backup copies of 
controlled software library (CM) 

• Establish User's Guide and System Description base- 
line (TM) 
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4.4.6 CONFIGURATION MANAGEMENT IN THE ACCEPTANCE TEST PHASE 


(CM=Conf iguration Manager, TL=Task Leader, TM=Technical 
Manager, DT=Development Team Member) 

Baselines Established 

• None 
Actions 

• Track requirements changes 

Document with a CCR form (TM, TL, or member of 
requirements team) 

Submit form to CMO (CM) 

Track change if approved into User's Guide and 
System Description 

• Track questions and answers between requirements team 

and developers 

Submit all questions and answers through CM 

Record question and date asked, and date an- 
swered when available, normally using CAT (CM) 

Follow up on any question unanswered for more 
than 2 weeks (CM) 

Track any resulting requirements changes from 
this process as described above 

• Track changes to controlled software library 

Prepare COF or equivalent and submit to CM for 
each newly coded module after it has been unit 
tested for addition to library (DT) 

Prepare CRFs for changes to coded modules to 
correct errors detected in acceptance testing 
and submit to CM (DT) 

Update library with change indicated on CRF and 
retain copies of CRF (CM) 

Provide appropriate number of backup copies of 
controlled software library (CM) 
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APPENDIX A - QUALITY ASSURANCE PLAN GUIDELINES 


This appendix presents guidelines for preparing the Quality- 
Assurance Plan, which is normally developed as part of the 
Software Development/Management Plan. The guidelines include 
quality assurance activities for each software product tailored 
for a typical attitude ground support system development ef- 
fort. Quality Assurance Plans for other types of projects 
must be tailored to the specific project. The general guide- 
line is to identify standards and a quality assurance process 
for each project deliverable to ensure that what is expected 
to be delivered is delivered (see Section 3.1). 

1.0 INTRODUCTION 

1 . 1 PURPOSE 

Explicitly state why the plan is needed and how it is intended 
to be used by all development team members. The plan is not 
only for the quality assurer; it also contains the quality 
assurance activities that affect the entire system development 
effort. For example: 

The purpose of the Quality Assurance Plan is to establish 
the quality assurance procedures for this project by iden- 
tifying the specific standards and guidelines to be used 
in the production of project documents and software. The 
plan is also to be used as a reference for all development 
team members for the standards and tools used during each 
phase of the development. 

1 . 2 GOALS 

Explicitly state what is to be achieved as a result of this 
plan, for example: 

The goal of this plan is to ensure that the quality assur- 
ance process is an integral part of the system development 
and that quality products are produced. 


0389 


A-l 


2.0 QUALITY ASSURANCE PROCEDURES 

Quality assurance procedures are product oriented. They will 
be applied at the onset of each product development. The 
stage must be set for the development of each product by iden- 
tifying the standards that are used as the guideline for the 
development and by ensuring that development team members 
understand the standards. After this is accomplished, the 
quality assurer must enforce the standards through appropriate 
spot checks and reviews and followup activity. 

In the Quality Assurance Plan, the procedures that will be 
used for each project product will be described as follows. 

2.1 QUALITY ASSURANCE PRODUCTS 

• List the products that will be monitored for quality 
assurance . 

• Discuss any major deviation from the list provided in 
Section 3.3 of this document. 

2.1.1 Quality Assurance Procedures for Documents 
For each document, discuss the following: 

• Schedule 

When does drafting of the document begin? 

When is the document to be delivered, both pre- 
liminary and final, if applicable? 

When will spot checks be made of the draft 
material for compliance with standards? 

When will the document be reviewed for compli- 
ance with the identified standard? 

Will a formal audit be made of this document? 

If so, when is it scheduled? 

Is this document scheduled for review by the 
technical publications department? If so, when? 
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Standards 


What standard format will be used to develop the 
document? 

Where is the standard referenced? 

Are any deviations from the standard allowed? 

What are the procedures for approval of devia- 
tions from the standard? 

Who has approval authority for deviations from 
the standard? 

• Checklists 

What checklists will be used in preparing the 
document or in determining that the document is 
complete? 

What are the procedures for deviations from the 
checklist, and who has approval authority for 
deviations? 

• Process 

Will quality assurance meetings be held to kick 
off the initiation of each new document? 

- What procedure will be followed for spot check- 
ing and reviewing the document? 

Will there be one-on-one encounters between the 
quality assurer and the writer for spot checks, 
or will the writer be asked to submit the drafts 
to the quality assurer for independent review? 

How will the formal reviews be conducted? Will 
the task leader and the technical manager be 
responsible for the final content of the docu- 
ment? Does the quality assurer play a role in 
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the review or does he/she only certify that the 
review has been made? 

2.1.2 Quality Assurance Procedures for Software 
For each software product, discuss the following: 

• Schedule 

When is the development of software components 
initiated? For example, when are program pro- 
logs and PDL first developed? When is coding 
begun? When is the system delivery file built? 

When are the components to be completed? De- 
scribe by phase of development and builds within 
the implementation phase. 

When are the quality assurance tools introduced 
for the software components? 

When are spot checks, reviews, and audits of the 
software components to be made? 

• Standards 

What standards will be used to guide the devel- 
opment of the software components? 

Where are the standards referenced? 

- Are any deviations from the standards allowed? 

What are the procedures for approving deviations 
from the standards? 

Who has approval authority for deviations from 
the standards? 

• Tools 

Will any automatic quality assurance tools be 
used? For example, will RXVP80 be used to ana- 
lyze FORTRAN code as an adjunct to unit testing? 
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Who will be responsible for making computer runs 
with the tools? 

What is the distribution of output from the 
tools? 

• Process 

How are spot checks made of the code? From 
developers' libraries before unit testing? 

When is the code reviewed? Before submission to 
the controlled software library? 

What is necessary for the review to be made? 
Source code listing and RXVP80 output? 

What happens when the code does not meet the 
standard? 

How is quality assurance approval achieved for 
each software component? What forms, what 
tools, who certifies, and who records? 

2 . 2 EXTERNAL INTERFACES 

• Will external organizations conduct formal audits? 

If so, they should be identified. 

2.3 ROLE OF QUALITY ASSURER 

Describe the specific duties of the quality assurer and how 
he/she will interface with the development team members. Dis- 
cuss the following: 

• Is the role of the quality assurer primarily as the 
tracker of quality assurance activities, or does 
he/she participate in the spot checks and reviews of 
the documents and the software? 

• Does the quality assurer have the authority to direct 
developers who are falling short of the standards? 
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Or does the quality assurer report to the task leader 
or the technical manager to have the standards 
enforced? 

• How does the quality assurer keep track of the qual- 
ity assurance spot checks, reviews, and audits? 

• When does the quality assurer know that enough spot 
checks have been made of either the document or soft- 
ware components to ensure standards compliance? 
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APPENDIX B - CONFIGURATION MANAGEMENT PLAN GUIDELINES 

This appendix presents guidelines for preparing the Configura- 
tion Management Plan, which is normally developed as part of 
the Software Development/Management Plan. The guidelines in- 
clude configuration management activities tailored for a typ- 
ical attitude ground support system development effort. 
Configuration Management Plans for other types of projects 
must be tailored to the specific project. Particular atten- 
tion is devoted to identifying the products to be placed under 
configuration management control, establishing the baselines 
for these products, and identifying the types of changes to be 
tracked to the product baselines (see Section 4.1). 

1.0 INTRODUCTION 

1 . 1 PURPOSE 

Explicitly state why the plan is needed and how it is intended 
to be used by all development team members. The plan is not 
only for the configuration manager; it also contains the con- 
figuration management activities that affect the entire system 
development effort. In addition, it includes the activities 
required to interface with external organizations such as the 
requirements team and the Configuration Control Board (CCB) . 
For example: 

The purpose of the Configuration Management Plan is to 
identify the products to be change management controlled, 
establish baselines for these products, define the types 
of changes to be tracked, and describe the configuration 
management procedures for each controlled product. 
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1 . 2 GOALS 


Explicitly state what is to be achieved as a result of this 
plan. For example: 

The goal of this plan is to ensure that the configuration 
management process is fully integrated into the develop- 
ment effort and fully understood by all development per- 
sonnel . 

2.0 CONFIGURATION MANAGEMENT PROCEDURES AND TOOLS 

Configuration management activities are software product 
oriented; the configuration management procedures and tools 
will be described as they relate to the products that are 
identified for configuration control. 

2.1 CONTROLLED PRODUCTS 

• List the products that will be controlled. 

• Discuss any major deviations from the list provided 

in Section 4 of this document, for example: "No Pre- 

liminary Design Report will be produced." or "A Data 
Base Description Document will be produced." 

2.1.1 Configuration Management Procedures for Documents 
For each document discuss the following: 

• Schedule 

When is the baseline created? Include phase of 
development and calendar target date if pos- 
sible, for example: "The Preliminary Design 

Report will be established as a baseline 5 days 
before the Preliminary Design Review." 

How long does the baseline remain under control 
and what document replaces it, if applicable? 
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Change 

What kinds of changes will be tracked to the 
baseline? For example: "Major requirements 

changes that result in major design/performance/ 
resource changes will be tracked. 

What forms will be used to track changes to the 
document baseline? 

Who has the approval authority for the change? 

Is this document under CCB control? If so, de- 
scribe the interface procedures with the CCB. 

Who is responsible for filling out the change 
form? 

Who is responsible for making the change? 

Where will the change forms be maintained? 

Will informal changes be allowed? If so, what 
are they? 

Process 

What is the procedure for establishing the base- 
line? 

What does the baseline contain? 

Where will the document baseline be kept, for 
example, as a PANVALET file or as a set of PC 
files, or both? 

What kinds of changes will be made to the base- 
line and who will initiate them, who will ap- 
prove them, and who will make them? What are 
the deviations from this process? 


How are the changes tracked after a change is 
initiated until it results in a document up- 
date? Are records kept of the changes after 
they have been made? 

2.1.2 Configuration Management 'Procedures for Software 
Products 

For software products, discuss the following: 

• Schedule 

When is the controlled software library baseline 
established? 

• Baseline 

Since software libraries are established at the 
beginning of the design phase of development, 
discuss the continuum of events from the design 
phase to the implementation phase when the con- 
trolled baseline is established as a controlled 
software library and the process for establish- 
ing the baseline. Identify the prefixes for 
major subsystems, utilities, and the naming con- 
ventions . 

What are the contents of the baseline by soft- 
ware type? 

What library management tools will be used to 
maintain the controlled software library(s)? 

• Change 

Where does the code to be modified reside? 

What change forms are required? 

Who reviews and/or approves the changes? 
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Where are tests of pending changes run before 
they are added to the controlled library? (Are 
they run from the user's library?) 

Who has read and/or write access to the con- 
trolled library? 

What is the procedure for failure recovery? 

Who creates execution modules for testing? 

How often will the controlled library be up- 
dated? Is it updated for every change, or is it 
updated when a batch of changes are available? 
How are emergency changes handled? 

What is the criteria for adding new items to the 
library? Is it the same for all types of items? 

Is system growth to be monitored? If so, what 
procedures will be followed to track growth? 

Who approves- the addition of new items to the 
library? 

What forms are used to add new items to the 
library? 

When is regression testing necessary? 

• Process 

How is the controlled software library baseline 
established? 

What steps do the developers take to add or make 
changes to the controlled library? Will the 
code be code read and unit tested and approved 
before it is added to the library? 
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How is the controlled library maintained from 
one system build to another? Is a separate li- 
brary established for each system build, or is a 
single library used throughout the development 
effort? 

What are the update procedures during integra- 
tion testing (build or release testing), system, 
and acceptance testing? Are there any differ- 
ences that need to be identified? 

• What are the backup and recovery procedures for 
the controlled software library? 

2 . 2 EXTERNAL INTERFACES 

• Requirements Team 

Are Question and Answer Forms used to track 
questions and answers between the requirements 
team and the developers? 

Who tracks the answers? 

Who determines the effect of the answers? 

What are the procedures for changes that result 
from the question-and-answer process? 

• Configuration Management Office (CMO)/Conf iguration 
Control Board (CCB) 

Who is the point of contact for CMO/CCB transac- 
tions? 

What are the internal procedures for tracking 
changes that are submitted to the CCB? 

Who are the Review Item Disposition Forms sent 
to and received from? 
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• What are the procedures for handling RID re- 
sponses? 

Who prepares the Document Change Notices? 

• Configuration Analysis Tool 
Who inputs the data? 

What is the frequency for each report? 

What is the distribution of the CAT reports? 

2.3 ROLE OF THE CONFIGURATION MANAGER 

Describe the specific duties of the configuration manager and 
how he/she will interface with the development team members. 
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APPENDIX C - QUALITY ASSURANCE CHECKLISTS 


The checklists presented in this appendix contain the informa- 
tion that will be covered for deliverable software development 
products that have been described in this document. The 
checklists are to be used as guidelines. For projects that do 
not follow the normal life-cycle development or do not produce 
the standard products, appropriate checklists will be devel- 
oped and included in the Quality Assurance Plan. The 
following checklists are presented: 

• Software Development/Management Plan Quality 
Assurance Checklist 

• Requirements Analysis Summary Report Quality 
Assurance Checklist 

• Preliminary Design Report Quality Assurance Checklist 

• Detailed Design Document Quality Assurance Checklist 

• PDL Standards Review Quality Assurance Checklist 

• FORTRAN Code Quality Assurance Checklist 

• Code Reading Quality Assurance Checklist 

• Build/Release/System Test Plan Quality Assurance 
Checklist 

• System Description Quality Assurance Checklist 

• User's Guide Quality Assurance Checklist 

• System Delivery File Quality Assurance Checklist 
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SOFTWARE DEVELOPMENT/MANAGEMENT PLAN QUALITY ASSURANCE 

CHECKLIST 


1. Purpose of project 

2. Relationship of project's software products to overall 
system 

3. Development team size, staffing pattern, and team composi- 
tion and organization 

4. External group interfaces, points of contact, and group 
responsibilities 

5. Development approach (technical and management) for every 
phase as follows: 

• Assumptions and constraints 

• Anticipated and unresolved problems 

• Major activities for the phase 

• List of development methods and tools 

• Products for the phase 

• Resource estimates (staff, computer, software reuse, 
documentation, software rehosting, and maintenance) 

• Definition of progress accounting in terms of 
metrics, data collection methods and forms, and sum- 
mary report forms 

6. Quality Assurance Plan 

7. Configuration Management Plan 

8. Major reviews (SRR, PDR, CDR) — Staff roles, schedule, 
attendees, materials 

9. Milestone charts 

• Development life-cycle phases 

• Delivery of required external interfaces 

• Schedule dates for integration of externally devel- 
oped software (if applicable) 

• Delivery of all development products 

• Internal reviews 

• Internal products 
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REQUIREMENTS ANALYSIS SUMMARY REPORT QUALITY ASSURANCE 

CHECKLIST 


1. Purpose of project 

2. Overall system concepts 

3. System description — Baseline diagrams, hardware inter- 
faces, external interfaces, data flow diagrams, operating 
scenarios with interfaces and data flow 


4 . 


5. 


6 . 


7. 


8 . 
9. 


System constraints 

• Hardware 

• Operating system limitations 

• Support software limitations 

Areas of concern and TBD requirements 

• Assessment of their effects on system size, required 
effort, cost, and schedules 

• Priority assignment 

System and subsystem requirements 

• • Level of importance 

• Completeness 

• Input (frequency, volume, coordinates, units, and 
timing) 

• Output (frequency, volume, coordinates, units, and 
timing) 

Data interfaces 

• Name; function, frequency, coordinates, units, data 
type 

• Organization (e.g., sequential), medium, record lay- 
outs, storage requirements 

Summary of reusable code 

Estimates — System size, required effort, cost, schedule 
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PRELIMINARY DESIGN REPORT QUALITY ASSURANCE CHECKLIST 


1. Purpose of project 

2. Overall system concepts 

3. System description — Baseline diagrams, hardware inter- 
faces, external interfaces, data flow diagrams, operating 

scenarios with interfaces and data flow 

4. System constraints and effects on design 

• Hardware 

• ' Operating system limitations 

• Support software limitations 

5. Areas of concern and TBD requirements 

• Assessment of their effects on system size, required 
effort, cost, and schedules 

• Priority assignment 

• Effects of assumptions on the design 

6. Critique of alternative designs 

7. Subsystem design descriptions 

• Tree diagrams expanded to two levels below the sub- 
system driver 

• Data flow diagrams for each processing mode 

• Processing related to operator-specified input and 

action: points of control, functions performed, nor- 

mal results, abnormal results, error processing and 
recovery 

8. System and subsystem resource requirements — Hardware, pe- 
ripheral space, memory 

9. Data interfaces 

• Name, function, frequency, coordinates, units, data 
type 

• Organization (e.g., sequential), medium, record lay- 
outs, storage requirements 

10. Summary of reusable code 
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DETAILED DESIGN DOCUMENT QUALITY ASSURANCE CHECKLIST 


1. Purpose of project 

2. Overall system concepts 

3. System description — Baseline diagrams, hardware inter- 
faces, external interfaces, data flow diagrams, operating 
scenarios with interfaces and data flow 

4. System constraints and effects on design 

• Hardware 

• Operating system limitations 

• Support software limitations 

5. Areas of concern and TBD requirements 

• Assessment of their effects on system size, _ required 
effort, cost, and schedules 

• Priority assignment 

• Effects of assumptions on the design 

6. Subsystem design description 

• Overall subsystem capability 

• Tree diagrams expanded to component level showing 
interfaces and control flow 

• Restrictions for each processing mode 

• Internal storage requirements for each processing mode 

• Detailed input and output specifications 

• List of all error messages including system response 
and operator action 

• Prologs and PDL for every subroutine 

• Processing related to operator-specified input and 

action: points of control, functions performed, nor- 

mal results, abnormal results, error processing and 
recovery 

7. System and subsystems resource requirements — Hardware, 
peripheral space, memory 

8. Data interfaces 

• Name, function, frequency, coordinates, units, data 
type 

• Organization (e.g., sequential), medium, record lay- 
outs, storage requirements 

9. Summary of reusable code 
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PDT, STANDARDS REVIEW QUALITY ASSURANCE CHECKLIST 


1. Use of subjects, verbs, and objects in all PDL statements 

2. Use of only standard PDL control structure keywords, re- 
lational attributes, and prepositions 

3. Special character precedes and follows all comments 

4. PDL contains all variables and external references defined 
in the prolog 

5. PDL nouns defined in the data dictionary 

6. PDL contains all constraints, restrictions, abnormal ter- 
minations, messages, etc., identified in the prolog 

7. Standard formats used for syntax for sequential, selection 
control, and iteration statements 

8. Straightforward logic 

9. Compound conditions standard and used only when necessary 

10. Embedded PDL starts with a page break 

11. PDL length of 1 page (60 lines) or less 

12. Each PDL statement begins with a standard special character 
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FORTRAN CODE QUALITY ASSURANCE CHECKLIST 


1. Use of parameters for 

• Computer-related characteristics (e.g., word and 
character sizes) 

• Design-dependent data attributes (e.g., table size 
and message length) 

• I/O device-oriented characteristics 

• Constants used to set flags and to reference data in 
lists and tables 

3. Variable names describe the function of the variable 

4. Data and character value definitions clear 

5. Data appropriately initialized 

6. Appropriate guidelines followed for 

a Module interfaces 

• Comments 

• Control processing 

• Loop processing 

• Code indentation 

• Statement format and labels 

• Computational processing 

• I/O processing 

• Hardware-dependent processing 

7. Common file data logically related 
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CODE READING QUALITY ASSURANCE CHECKLIST 


1. All constants defined 

2. All unique values explicitly tested for input parameters 

3. Values stored after they are calculated 

4. Path defined for every possible outcome of a logical deci- 
sion 

5. All input defaults explicitly tested 

6. All unique values of a keyword checked 

7. Specified precision sufficient for required accuracy 

8. Flagged/missing data values correctly excluded from compu- 
tations 

9. Display formats large enough 

10. Data sets properly opened and closed 

11. Leading and trailing records recognized and handled appro- 
priately 

12. Unit numbers unique 

13. Double-precision constants used in double-precision ex- 
pressions 

14. Correct units or the appropriate conversion used (e.g., 
degrees, radians) 

15. Correct sign (positive or negative) used, especially in 
bias adjustments 

16. All counters properly initialized (0 or 1) 

17. Absolute and symbolic (parameter) values used appropriately 

18. On comparison of two bytes, all bits are compared, if nec- 
essary 

19. Variable names informative 

20. Prolog defines all variables used 

21. Code indented to show its structure 

22. Code adequately commented 

23. Prolog defines operational constraints and assumptions 

24. Continuation lines clearly indicated 

25. Statement labels occur in an obvious sequence 
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1. Purpose of test plan 

2. Type and level of testing 

3. Testing schedule 

4. Test description for every test 

• Specific capability or requirement tested 

• Detailed specification of input 

• Required environment (e.g., data sets and hardware) 

• Operational procedure 

• Detailed specification of output 

• Test result acceptance criteria 
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SYSTEM DESCRIPTION QUALITY ASSURANCE CHECKLIST 


1. Purpose of project 

2. Overall system concepts 

3. System description — Baseline diagrams, hardware inter- 
faces, external interfaces, data flow diagrams, operating 
scenarios with interfaces and data flow 

4. Subsystem description 

• Overall capability 

• Assumptions and restrictions to processing 

• High-level subsystem data flow diagrams 

• Input and output 

• Tree diagrams to component level 

5. System initiation procedures 

• Resource requirements with high level diagrams and 
tables for the system and subsystem for hardware, 
support data sets, peripheral space, memory, CPU 
time, I/O time 

• Specific initiation information 

• Program structure information (e.g., overlaying or 

loading control) 

6. Detailed description of input and output for each step 
(source code libraries, object code libraries, execution 
code libraries, etc.) 

7. Internal storage requirements — Description and size of 
arrays 

8. Data interfaces 

• Name, function, frequency, coordinates, units, data 

type 

• Organization, medium, record layouts, storage re- 
quirements 

9. Prologs and PDL 

10. List and description of components from support data sets 
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1 . 
2 . 
3 . 



Purpose of project 
Overall system concepts 

System description — Baseline diagrams, hardware inter- 
faces, external interfaces, data flow diagrams, operating 
scenarios with interfaces and data flow 

4. Subsystem description 

- • Overall capability 

• Assumptions and restrictions to processing 

• High-level subsystem data flow diagrams 

• Input and output 

• Description of subsystem input and output 

• Processing related to operator-specified input and 

action: points of control, functions performed, nor- 

mal results, abnormal results, error processing and 
recovery 

5. Execution requirements — Hardware, peripheral space, mem- 
ory, control information for each processing mode, timing 

considerations (CPU time, I/O time, wall-clock time) 

6. System and subsystem input and output description 

• Facsimiles of graphic displays in the order they are 
produced 

• Facsimiles of hardcopy output in the order produced 
and annotated to show controlling parameters 

• List of numbered error messages with system response 
and operator action 
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SYSTEM DELIVERY FILE QUALITY ASSURANCE CHECKLIST 


1. First file (description and installation guide) contains 

• System name 

• Abstract of system function 

• List of all files on the tape with file number and 
name and description of file contents for every file 

• Operating environment (hardware and software require- 
ments) 

• System unloading procedure 

• Supporting documentation references 

2. All files to generate an execution load module include 

• Primary source code 

• Command procedures (e.g., JCL) to compile and link- 
ed i t the source code 

• Subroutine libraries 

3. Every file to execute the system contains 

• Load module library 

• Support libraries 

• Command procedures (e.g., JCL) to execute the system 

• Permanent input and output data sets, including re- 
quired initialization routines 

4. Every file to test the system contains 

• Command procedures (e.g., JCL) to execute the system 
for each test 

• Permanent input data sets for each test 

• Output data for each test 
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APPENDIX D - STANDARD CONFIGURATION MANAGEMENT CHANGE FORMS 


The forms presented in this appendix are used to record 
changes during the normal development process as described in 
this document. If new and different forms are necessary for a 
specific project, they will be included in the Configuration 
Management Plan. The following forms are presented: 

• Review Item Disposition (RID) 

• RID Response 

• Requirements/Specifications Question and Answer Form 

• Configuration Change Request (CCR) 

• Document Change Notice (DCN) 

• Component Origination Form (COF) 

• Change Report Form (CRF) 
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RID NO 


FL I G5I— IT 
DVNAM I OS 
D I V I S I OINI 


REV I SW ITEM 
D I S«=-OS I T X OlxJ 
< RID ) 


DATE RECEIVED 


RECEIVED BY 


REVIEW DATE 



DISPOSITION: □ APPROVED BY ORIGINATOR □ DEFERRED TO CCB D WAIVER REQUESTED 


NASA RANA6ER 


SPONSORED BY: 


FDD/CWKCSO/OU (2/86) 


Review Item Disposition (RID) 
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RESPONSE NO. 


f=L_ I GHT RIO 

D I CS 

O I V I S I O INI RESROMSE 

I RECEIVED fir 


RESPONSE HATE 


RESPONDENT’S RECOMEND AT IONS 


REQUIRED DDCUHENT MANSES (ATTACKED) 

SECTION PA6E SECTION PASE SECTION PA5E 



RESPONDENT I ORGANIZATION 1 TELEPHONE 


RESPONSE APPROVAL: 

: □ APPROVED □ REJECTED 

ORIGINATOR 

DATE 

NASA HANA6ER 

DATE 


F0D/CK0(CSC)/012 (2/B6) 


RID Response 


RID NO. 


DOH'NENT TITLE 


DATE RECEIVED 
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REQUIREMENTS/SPECIFICATIONS QUESTION AND ANSWER FORM 


NUMBER I 


Date Submitted ! 

Date Answered ! „ 

Submitted By Z _ 

Answered By Z 

Date Answer Needed Z 

Resulting CCR t Z ..... 

Affected Subsystems 1 . . _ 

Approved By 1 




QUESTION/COMMENT/RECOMMENDATION : 


Continuation Attached □ 


RESPONSE : 


Continuation Attached □ 


I 

I 

I 


Requirements/Specif ications Question and Answer Form 
(1 of 2) 
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I 

fl 


REQUIREMENTS/SPECIFICATIONS QUESTION 


NUMBER I 


CONTINUATION 


QUESTION/ COMMENT /RECOMMEND AT ION : 


Requirements/Specifications Question and Answer Form 
(2 Of 2) 
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FLIGHT DYNAMICS DIVISION CONFIGURATION CHANGE REQUEST (CCR) 


4. CHANGE LEVEL 

Di 03 
□ 2 04 

5. TITLE OF CHANGE 


6 . 

DOCUMENT TITLE 


1. CCR NO. 

2. CONFIGURATION ITEM NAME 

3. PRIORITY 
D EMERGENCY 
□ URGENT 



□ ROUTINE 


DOCUMENT NO* 


7. REASON FOR CHANGE 


8* DESCRIPTION Of CHANGE 


9. I HR ACT 

□ SCHEDULE 

□ BUDGET 

O FACILITIES 
O TESTING 

□ TRAINING 

□ CONTRACTOR SUPPORT 

□ INTERFACES 

10. COMMENTS 


ORGANIZATION j TELEPHONE 

12. AUTHORIZING SIGNATURE 


branch head date 

FDD/Cft0(CSC)/026 (4/86) 


Configuration Change Request (CCR) 


11. ORIGINATOR 


□ RELIABILITY D GROUND SEGMENT . □ 331 

O MAINTAINABILITY O SPACE SEGMENT □ 332 

□ USER SERVICES O LOGISTICS D 333 

□ SECURITY □ DOCUMENTATION □ 554 

□ USAF FUNDING REDD □ HARDWARE □ OTHER 

□ POWER O SOFTWARE 

□ WEIGHT □ OTHER 


I 

I 

I 
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FLIGHT 

DYNAMICS 

DIVISION 



DCN NUMBER 

DOCUMENT CHANGE NOTICE 
(DCN) 

DATE 



CHANGE REQUEST REFERENCE 

ORIGINATOR NAME 

ORGANIZATION 

TELEPHONE 

DOCUMENT TITLE 

DOCUMENT NUMBER 


PAGES CHANGED (INCLUDING DELETIONS) 


I 

I 

I 

I 


CONTINUED □ 


CMO SIGNATURE 


DATE 


CSC/OSSO/CMO/OI 9(5/85) 


Document Change Notice (DCN) 
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COMPONENT ORIGINATION FORM 


Component 

Programmer 

Subsystem 

Date _ _ ..... ... _ 

Project 


Location of source file 


Library or directory 

Member name 

Relative difficulty of component 


Please indicate (your judgement) by marking an 

X on the line below: 

Easy Medium Hard 


Origin 


If the component was modified or derived from a different project, please indicate the approx- 
imate amount of change and from where it was acquired; if it was coded new (from detailed 
design) indicate NEW. 

NEW 

Extensively modified (more than 25% o 
Sliahtlv modified 
Old (Unchanged) 

f state. i .ents changed) 

If not new r where is it from ? 


Type of Component 


'INCLUDE' file (e.g.. COMMON) 

Namelists or parameter lists 

JCL (or other control) 

Display identification (GESS) 

. ALC (assembler code) 

Menu definition or help 

FORTRAN executable source 

Reference data files 

Pascal source 

BLOCK DATA file 

Ada source 

other (describe) 

Purpose of Executable Component 


For executable code, please identify the major purpose or purposes of this component. (Check 
all that apply). 

I/O processina 
Algorithmic/ comDutational 
Data transfer 
Loqic/decision 
Driver module 

Interface to operating system 



GSFC 550-1 13/85) 

Component Origination Form (COF) (1 of 2) . 


I 

I 

I 
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Instructions for completing the Component Origination Form 


This form is completed by the programmer for each source code component, when it is ready to 
be added to the controlled library (that is, placed under configuration controlh There will be one 
CO form for each distinct component in the library. (For changes to components in the 
controlled library, use Change Report Forms.) 

Header Information: self-explanatory. If you do not know the names used for project and 
subsystem, see the task leader or the configuration manager. The date is the date that the 
form is completed— that is, after all the coding is complete. 

Location of Source: provide enough information for the librarian to move the file from your 
library into the controlled project library. This includes the device, the directory (on DEC machines) 
or library (on IBM), and the member or file name. 

Difficulty: use your own judgement. The continuum is provided that you may distinguish 
among several components that you develop. 

Origin: if the component is new (not copies from an earlier component) mark NEW. If you 
used pre-existing code, indicate how much you changed the old code and where you borrowed 
it from. 

Type of Component: select the one description that best typifies this component. HELP files 
are classed with MENU files. Reference data includes constants or parameters that are not subject 
to modification (format definitions . . .). 

Purpose of Executable Component: select one or more functions that describe the major 
purpose(s) of this component. 


Component Origination Form (COF) (2 of 2) 
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ORIGINAL PAGE IS 

OF POOR QUALITY 


CHANGE REPORT FORM 


project name 

PROGRAMMER NAME 


CURRENT DATE 
APPROVED BY _ 


SECTION A - IDENTIFICATION 


DESCRIBE THE CHANGE: (What, why, how) . 



EFFECT: What components ior documents) are changed? (Include version). 


EFFORT: What additional components (or documents) were examined in determining what change was needed? 


Need lor change determined on 


Change completed (incorporated into system) , 



Effort in person time to isolate the change (or error) 

Effort in person time to implement the change (or correction) 


I hr/less 1 hr/1 dy ldy/3dys >3dyt 



SECTION B - ALL CHANGES 


TYPE OF CHANGE (Check one) 


EFFECTS OF CHANGE 


□ Error correction 

O Binned enhancement 
G Implementation of requirements change 

□ Improvement of clarity, maintainability, 
or documentation 

O Improvement of user services 


O Inseriion/deletion of debug code 

□ Optimization of time/space/accuracy 
D Adaptation to environment change 

□ Other (Explain on back) 


□ □ Was the change or correction to one and 

only one component? 

□ □ Did you look at any other component? 

□ □ Did you have to be aware of parameters passed 

explicitly or'implicitly (e.g., common blocks) 
to or from the changed component? 


SECTION C - FOR ERROR CORRECTIONS ONLY 


SOURCE OF ERROR 
(Check one) 


□ Requirements 

O Functional specifications 

□ Design 


□ Previous change 


CLASS OF ERROR 
(Check most applicable)* 


□ initialization 

□ logic /control structure 

(e.g., flow of control incorrect) 

□ Interlace (internal) 

(module to module communication) 

□ Interface (external) 

(module to external communication) 

□ Oita (value or structure) 

(e.g., wrong variable used) 

□ Computational 

(t.g., error in math expression) 


mm 


CHARACTERISTICS 
(Check Y or N tor all) 


Y N 

□ □ Omission error (e.g., something was (aft out) 

O G Commission arror (e.g., something incorrect was included) 

□ □ Error was created by transcription (clerical) 


FOR LIBRARIANS USE ONLY 


NUMBER 

DATE : 

BY 

CHECKED BY 

(Month 0»y Year) 

ORIGIN DATE 


{Additions/ Comments on Reverse Side) 


Change Report Form (CRF) 
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